When Optimal Entropy-constrained Quantizers Have a Finite or Innnite Number of Codewords

An entropy-constrained quantizer Q is optimal if it minimizes the expected distortion D(Q) subject to a constraint on the output entropy H(Q). In this paper we use the Lagrangian formulation to show the existence and study the structure of optimal entropy-constrained quantizers that achieve a point on the lower convex hull of the operational distortion-rate function D h (R) = inf Q fD(Q) : H(Q) Rg. In general, an optimal entropy-constrained quantizer may have a countably innnite number of code-words. Our main results show that if the tail of the source distribution is suuciently light (resp. heavy) with respect to the distortion measure, the Lagrangian-optimal entropy-constrained quantizer has a nite (resp. innnite) number of codewords. In particular, for the squared error distortion measure, if the tail of the source distribution is lighter than the tail of a Gaussian distribution, then the Lagrangian-optimal quantizer has only a nite number of codewords, while if the tail is heavier than that of the Gaussian, the Lagrangian-optimal quantizer has an innnite number of codewords.